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Abstract 


Capability-based  acquisition  has  led  to  the  simultaneous  development  of  systems 
that  must  eventually  interact  within  a  system-of-systems  (or  major  sub-systems  that 
must  integrate  on  a  single  platform).  The  necessary  interdependencies  between 
systems  also  generate  complexity  and  can  increase  development  risk.  Trades 
between  capability  and  risk  are  essential  during  analysis  of  alternatives  in  pre¬ 
acquisition  phases.  For  example,  while  legacy  assets  can  potentially  provide  a 
certain  level  of  capability  with  relatively  low  risk,  their  eventual  capability  may  be 
restricted  because  of  some  specific  characteristic  or  inherent  rigidity.  These  features 
create  a  trade-off  space  between  development  risk  and  capability  potential  of  a 
system.  Existing  tools  for  such  trades  can  be  cumbersome  and  non-intuitive  when 
complexity  is  high.  The  authors’  prior  work  has  developed  a  Computational 
Exploratory  Model  to  simulate  the  development  process  dynamics  for  these  complex 
networks  of  systems  intended  for  a  system-of-systems  capability.  The  progress 
documented  in  this  paper  couples  the  computational  model  with  a  capability  module 
applied  to  the  Airborne  Laser  (ABL)  system  and  presents  an  exemplary  analysis  of 
alternatives  by  comparing  expected  development  time  and  capability  level  under 
certain  probabilities  of  disruption. 

Introduction 

The  purpose  of  capabilities-based  acquisition,  as  described  by  Charles  and  Turner 
(2004),  is  to  acquire  a  set  of  capabilities  instead  of  acquiring  a  family  of  threat-based, 
service-specific  systems.  The  Missile  Defense  Agency  (MDA),  for  example,  uses  capability- 
based  acquisition  to  evaluate  the  success  of  a  program  based  on  its  ability  to  provide  a  new 
capability  for  a  given  cost,  and  not  on  its  ability  to  meet  specific  performance  requirements 
(Spacy,  2004).  The  Joint  Mission  Capability  Package  (JMCP)  concept  is  another  example 
that  aims  to  create  a  joint  interdependency  between  systems  to  combine  capabilities  in 
order  to  maximize  reinforcing  effects  and  minimize  vulnerabilities  (Durkac,  2005).  The  goal 
is  a  more  efficient  utilization  of  both  human  and  machine-based  assets  and,  in  turn, 
improved  combat  power.  In  these  settings,  systems  are  increasingly  required  to 
interoperate  along  several  dimensions,  which  characterizes  them  as  systems-of-systems 
(SoS;  Maier,  1998).  SoS  most  often  consist  of  multiple,  heterogeneous,  distributed  systems 
that  can  (and  do)  operate  independently  but  can  also  assemble  in  networks  and  collaborate 
to  achieve  a  goal. 
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The  presence  of  interdependencies  in  layered  networks  spanning  a  hierarchy  of 
levels  is  one  of  the  sources  of  complexity  in  SoS  development  (DeLaurentis  et  al.,  2008a, 
2008b;  Ayyalasomayajula  et  al.,  2008;  Kotegawa  et  al.,  2008).  The  interdependencies 
between  component  systems  often  result  in  complex  networks  that  exhibit  vulnerabilities  to 
disruptions  in  the  development  of  even  one  system,  especially  if  that  one  system  places  a 
central  role  in  the  network.  Gell-Mann  (2002)  defines  complexity  as  the  amount  of 
information  necessary  to  describe  regularities  of  a  system  effectively.  Rouse  (2001 ) 
summarizes  the  complexity  of  a  system  (or  model  of  a  system)  as  related  to  the  intentions 
with  which  one  addresses  the  system,  the  characteristics  of  the  representation  that 
appropriately  accounts  for  the  system’s  boundaries,  architecture,  interconnections,  and 
information  flows,  and  the  multiple  representations  of  a  system.  We  can  represent  degrees 
of  complexity  by  examining  the  graphs  that  result  when  we  record  the  intentions, 
characteristics,  interconnections,  etc.,  in  a  given  situation. 

Acquisition  programs  have  struggled  with  complexities  in  both  program  management 
and  engineering  design  (e.g.,  NASA’s  Constellation  Program  [Committee  on  Systems 
Integration  for  Project  Constellation,  2004]  and  FAA’s  NextGen  [NextGen  Integration  and 
Implementation  Office,  2009]).  While  first-order  impacts  of  decisions  are  nearly  always 
considered,  the  cascading  effects  that  result  from  complex  interdependencies  obscure  the 
quantification  and  visibility  of  the  higher-order  impacts  of  developmental  decisions  and 
disruptions.  Furthermore,  the  network  structure  behind  the  collaboration  can  contribute  both 
negatively  and  positively  to  the  successful  achievement  of  SoS  capabilities  and,  even 
earlier,  to  the  developmental  success.  Collaboration  via  interdependence  may  increase 
capability  potentials,  but  it  also  contains  concealed  risk  in  the  development  and  acquisition 
phases. 

Our  approach  quantifies  the  impact  of  system  interdependencies  in  the  context  of 
system  development  and  capability.  It  provides  a  means  to  conduct  analysis  of  alternatives 
while  navigating  the  decision  space  that  simultaneously  considers  the  potential  positive 
impacts  of  interdependencies  (e.g.,  capability)  as  well  as  the  negative  impacts  (e.g., 
development  time).  The  work  comprises  new  improvements  to  a  Computational  Exploratory 
Model  (CEM) — a  discrete  event  simulation  model — previously  introduced  in  prior  Acquisition 
Symposia  (Mane  and  DeLaurentis,  2009,  2010)  that  aims  to  provide  decision-makers  with 
insights  into  the  development  process  by  propagating  development  risk  in  the  SoS  network. 
The  impact  that  system  risk,  system  interdependencies,  and  system  characteristics  have  on 
the  estimated  completion  of  a  program  are  generated.  We  present  a  proof-of-concept 
application  that  analyzes  the  development  time  of  the  Airborne  Laser  (ABL)  system  and 
conduct  a  trade-off  study  between  development  time  and  capability  while  considering 
various  alternatives  for  the  constituent  systems  of  the  ABL. 

Computational  Exploratory  Model  (CEM)  Overview 

The  CEM  is  based  on  the  16  basic  technical  management  and  technical  system¬ 
engineering  processes  outlined  in  the  Defense  Acquisition  Guidebook  (DoD,  2008a),  often 
referred  to  as  the  5000-series  guide.  However,  an  SoS  environment  changes  the  way  these 
processes  are  applied.  The  Systems  Engineering  Guide  for  System-of-Systems  (SoS-SE; 
DoD,  2008b)  addresses  these  considerations  by  modifying  some  of  the  16  processes  in 
accord  with  an  SoS  environment.  The  resulting  processes  and  respective  functions  consist 
of  translating  inputs  from  relevant  stakeholders  into  technical  requirements,  developing 
relationships  between  requirements,  designing  and  building  solutions  to  address 
requirements,  integrating  systems  into  a  high-level  system  element,  and  performing  various 
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managing  and  control  activities  to  ensure  that  requirements  are  effectively  met,  risks  are 
mitigated,  and  capabilities  achieved. 

The  CEM,  centered  on  these  revised  processes,  is  a  discrete  event  simulation  of  the 
development  and  acquisition  process.  This  process  creates  a  hierarchy  of  analysis  levels: 
SoS  Level  (LI),  Requirement  Level  (L2),  and  System  Level  (L3).  Component  elements  at 
each  level  are  a  network  representation  of  the  level  below.  The  SoS  Level  (LI)  is  comprised 
of  the  numerous,  possibly  interdependent  requirements  (L2)  needed  to  achieve  a  desired 
capability.  Similarly,  satisfaction  of  each  requirement  in  the  Requirement  Level  (L2)  requires 
a  number  of  possibly  interdependent  systems  (L3). 

At  the  Requirement  Level  (L2),  Requirements  Development  contains  the  technical 
requirements  of  the  SoS  (provided  externally).  The  technical  requirements  are  then 
examined  in  Logical  Analysis  to  check  for  interdependencies  among  the  requirements.  A 
check  for  inconsistencies  among  requirements  is  also  performed.  Design  Solution 
development  and  Decision  Analysis  are  the  next  processes,  which  belong  to  the  System 
Level  (L3).  They  produce  the  optimal  design  solution  from  the  set  of  feasible  solutions  to 
meet  the  given  requirements.  The  optimal  design  solution  not  only  is  based  on  the  current 
set  of  requirements  and  solution  alternatives  but  also  takes  into  account  all  previous 
information  available  through  requirements,  risk,  configuration,  interface,  and  data 
management  processes.  Because  most  acquisitions  are  multi-year  projects  involving  many 
different  parties,  the  overlap  between  the  management  processes,  Design  Solution  and 
Decision  Analysis,  allows  for  greater  tractability  of  decisions.  It  is  at  this  stage  that  system 
interdependencies  are  identified.  The  optimal  design  solution  obtained  from  this  phase  is 
then  sent  to  the  next  stage:  Technology  Planning  and  Technology  Assessment.  In  the  event 
that  an  optimal  or  sub-optimal  design  solution  to  successfully  implement  the  given 
requirements  does  not  exist,  the  feedback  loop  to  Requirement  Development  translates  into 
a  change  in  the  technical  requirements  for  the  SoS.  Technology  Planning  and  Technology 
Assessment  are  System  Level  (L3)  scheduling  processes  that  oversee  the  implementation, 
integration,  verification,  and  validation  for  all  the  component  systems  in  the  SoS. 

The  Implementation  and  Integration  Phases  of  component  systems  constitute  the 
lowest  level  of  detail  modeled  in  the  CEM.  The  design  decisions  made  at  earlier  stages 
must  be  implemented  and  integrated  in  these  phases  to  generate  the  final  product  of  a 
program.  Figure  1  presents  an  abstraction  of  the  layered  networks  that  result  from  the 
modeling  of  the  acquisition  process:  systems  are  grouped  to  satisfy  a  requirement,  and 
requirements  are  grouped  to  generate  a  capability. 


^  SoS  Capability 


Systems 

(system  capability) 


Figure  1.  Layered  Network  Abstraction  of  Computational  Exploratory  Model 
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Systems  can  be  independent,  can  satisfy  several  requirements,  and  can  depend  on 
other  systems.  The  CEM  simulates  these  layered  relationships  to  capture  the  impacts  that 
any  changes — related  to  decision-making,  policy,  or  development — in  any  of  the  component 
systems,  requirements,  and  relationships  between  them  have  on  the  completion  of  a  project. 
In  our  prior  experiments,  we  studied  the  impact  of  different  interdependency  topologies.  The 
exercise  of  the  CEM  described  in  this  paper  assumes  a  fixed  topology  and  instead 
specifically  targets  variations  in  inherent  system  risk,  the  interdependency  strength  among 
systems,  and  the  span-of-control  of  the  SoS  authority  (if  present).  The  next  section  presents 
the  CEM  model  dynamics  and  input  parameters. 

Model  Input  Parameters 

The  CEM  operates  as  a  discrete  event  simulator  of  the  development  process.  It 
models  risk  (probability  of  a  disruption  and  associated  consequence)  present  in  the 
implementation  and  integration  of  each  component  system  as  well  as  the  risk  due  to  the 
system  interdependencies.  Furthermore,  systems  and  SoS  engineers  are  often  faced  with 
the  decision  of  using  legacy  assets  to  satisfy  a  given  requirement  or  opt  for  the  development 
of  brand  new  ones.  The  CEM  includes  parameters  such  as  readiness-level  to  differentiate 
between  legacy  assets/platforms,  new  systems,  and  partially  implemented/integrated 
systems  (i.e.,  systems  under  development)  and  to  investigate  the  impact  that  the  inclusion 
of  such  systems  in  the  development  of  an  SoS  has  on  the  success  of  a  project.  Table  1 
presents  the  input  parameters,  and  the  remainder  of  this  section  expands  and  explains  their 
role  in  the  CEM. 
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Table  1. 


Input  Parameters  of  Computational  Exploratory  Model 


Parameter  Notation  Description 


Requirement  Level  (L2) 


Requirement 

dependencies 

Dreq 

Adjacency  matrix  that  indicates  requirement 
interdependencies 

Risk  profile 

Rreq 

Probability  of  disruptions  in  Requirement  Development 

Phase 

Impact  of  disruptions 

lreq 

Time  penalty  when  disruptions  hit  Requirement 

Development  Phase 

System  Level  (L3) 

System  dependencies 

Dsys 

Adjacency  matrix  that  indicates  system  interdependencies 

Development  pace  of 
design 

Ides 

Increase  in  completion  of  Design  Solutions  Phase 

Design  risk  profile 

Rdes 

Probability  of  disruptions  in  Design  Solutions  Phase 

Impact  of  design 
disruptions 

Ides 

Time  penalty  when  disruptions  hit  Design  Solutions  Phase 

Span-of-control 

soc 

Indicator  of  how  Implementation  and  Integration  are 
performed  (sequentially  or  simultaneously) 

System  initial  readiness- 
level 

m°(i,r) 

Initial  readiness-level  of  system  /'  to  satisfy  requirement  r  (for 
Implementation  Phase) 

System  risk  profile 

RSys(i,r) 

Probability  of  disruptions  (during  implementation)  of  system  /' 
when  satisfying  requirement  r 

Impact  of  disruptions 

lsys{l) 

Time  penalty  when  disruptions  hit  system  /'  during 
Implementation/ Integration 

Implementation  pace 

Pimp(l) 

Increase  in  readiness-level  at  each  time-step  during 
implementation  of  system  / 

Integration  pace 

Pint(l) 

Increase  in  completeness-level  at  each  time-step  during 
integration  of  system  / 

Implementation  start 

limp{IJ) 

Readiness-level  of  system  j  when  Implementation  Phase  of 
dependent  system  /'  begins 

Strength  of  dependency 

S(IJ) 

Strength  of  dependency  of  system  /'  on  system  j 

The  requirement  dependency  matrix  ( Dreq )  indicates  how  the  development  and 
satisfaction  of  requirements  depend  on  each  other,  which  impacts  the  sequence  in  which 
requirements  are  developed  and  satisfied.  For  example,  if  Requirement  A  depends  on 
Requirement  B,  then  development  of  Requirement  A  begins  when  Requirement  B  has  been 
satisfied.  As  requirements  are  developed,  the  risk  profile  ( Rreq )  of  Requirement 
Development  indicates  the  probability  of  disruptions  at  this  stage  in  the  development 
process.  Disruptors  signify  a  change  in  requirements  or  the  addition  of  new  requirements. 
When  a  requirement  is  changed  after  the  acquisition  process  has  begun,  it  affects  all 
subsequent  processes  and  causes  a  time  delay  ( lreq )  that  is  added  to  the  project  time.  Every 
requirement  that  is  implemented  is  fed  into  its  own  Design  Solution  and  Decision  Analysis 
process.  The  Design  Solution  and  Decision  Analysis  processes  feed  into  each  other,  and 
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the  risk  profile  ( Rdes )  indicates  the  probability  of  disruptions  at  each  time-step  during  the 
completion  of  the  stage  with  a  value  between  0  and  1 .  Any  disruptions  at  this  stage  indicate 
that  the  design  solution  provided  is  not  feasible  and  a  time  penalty  ( ldes )  that  indicates  a  re¬ 
design  of  the  solution  is  incurred.  If  the  solution  fails  in  multiple  consecutive  time-steps, 
then  the  requirement  is  sent  back  to  the  Requirement  Development  stage;  otherwise,  the  set 
of  component  systems  and  their  user-defined  parameters  are  sent  to  the  Technical  Planning 
and  Technical  Assessment  processes,  based  on  the  development-pace  parameter  of  this 
stage. 

The  Implementation  Phase  simulates  the  development  of  each  system.  The  nature 
of  candidate  systems  may  range  from  legacy  systems  to  off-the-shelf,  plug-and-play 
products  to  custom-built,  new  systems.  Here,  we  define  legacy  systems  as  systems  that 
have  been  developed  in  the  past  to  achieve  a  particular  requirement,  and  new  systems  as 
not-yet-developed  systems  envisioned  to  satisfy  a  new  requirement.  When  considering  the 
use  of  legacy  systems  to  meet  a  new  requirement,  the  capability  of  these  systems  to  satisfy 
the  new  requirement  is  not  necessarily  the  same  as  their  capability  to  meet  the  original 
requirement  for  which  they  were  designed.  Additionally,  the  risk  associated  with  the 
modification  of  a  legacy  system  and  the  risk  associated  with  the  development  of  a  brand 
new  system  can  be  quite  different.  Legacy  systems  may,  however,  provide  cost  and/or  time 
benefits  if  modifications  are  less  severe  than  a  new  development,  as  is  the  case  with  new 
systems.  To  delineate  systems  in  a  meaningful  way,  we  describe  the  spectrum  of  a 
system’s  ability  to  satisfy  a  requirement  in  terms  of  its  readiness-level. 

System  readiness-level,  a  concept  proposed  by  Sauser  et  al.  (2006),  is  a  metric  that 
incorporates  the  maturity  levels  of  critical  components  and  their  readiness  for  integration 
(i.e.,  integration  requirements  of  technologies).  This  is  an  extension  of  the  widely  used 
Technology  Readiness-Level  (TRL),  a  metric  that  assesses  the  maturity  level  of  a  program’s 
technologies  before  system  development  begins  (USD[AT&L],  2005).  While  similar  in  spirit 
to  the  SRL  metric  proposed  by  Sauser  et  al.  (2006),  readiness-level  in  the  present  work  is 
defined  in  a  different  manner  and  with  less  detail.  We  define  system  readiness-level  as  the 
readiness-level  of  a  system  /'to  satisfy  requirement  r,  m(i,r),  with  a  value  between  0  and  1 . 

A  system  with  a  readiness-level  of  1  is  a  fully  developed  system  that  can  provide  a  certain 
level  of  capability.  The  dynamic  model  starts  the  Implementation  Phase  of  a  system  from  its 
initial  readiness-level  and  simulates  its  development/implementation  until  it  reaches  a 
readiness-level  of  1 .  An  initial  readiness-level  of  0  indicates  a  brand  new  system  that  must 
be  developed  from  scratch,  while  a  system  with  an  initial  readiness-level  greater  than  0 
indicates  a  legacy  system  that  is  partially  developed  to  satisfy  a  requirement  r  but  needs 
further  development  to  reach  a  readiness-level  of  1 .  In  general,  careful  research  of  a 
candidate  system  /'  will  determine  its  initial  readiness-level  to  satisfy  a  requirement  r,  and, 
therefore,  the  amount  of  development  necessary  to  achieve  a  readiness-level  of  1 .0. 

The  CEM  simulates  the  Implementation  Phase  as  a  series  of  time-steps  in  which  a 
pre-determined  increment  of  readiness  (p,mp(0)  is  gained  at  each  time-step  of  each  system  / 
or  lost  if  a  disruption  occurs  (according  to  the  system  risk  profile  of  system  /  in  satisfying 
requirement  r,  Rsys(i,r))-  This  is  clearly  a  gross  simplification  of  the  actual  development 
process  for  a  system;  however,  it  adequately  serves  the  purposes  of  the  research,  which  is 
focused  on  the  interdependencies  between  systems  to  develop  a  SoS  capability  and  aims  to 
capture  the  impact  of  disruptions  on  the  development  process.  Accurate  modeling  of  the 
Implementation  Phase  would  increase  the  accuracy  of  the  model  for  a  particular  application, 
but  it  would  not  change  the  nature  of  the  observed  results. 
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Representation  of  Disruptions 

The  risk  associated  with  the  development  of  a  system  is  a  function  of  its  inherent 
characteristics  (technology,  funding,  and  complexity  levels)  and  on  risk  levels  of  the  systems 
on  which  it  depends.  The  former  may  be  estimated  via  a  variety  of  analysis  techniques  that 
examine  a  system  in  detail,  but  the  latter  requires  knowledge  of  system  interdependencies 
that  can  be  numerous,  complicated,  and  often  opaque.  Developmental  interdependencies 
of  SoS  create  layered  networks  that  often  span  among  a  hierarchy  of  levels  (DeLaurentis  et 
al.,  2005;  Butler  et  al.,  2001;  Ayyalasomayajula  et  al.,  2008;  Kotegawa  et  al.,  2008).  The 
complexity  of  these  networks  often  hides  many  of  the  otherwise  explicit  consequences  of 
risk.  Depending  on  the  network  topology  characteristics,  disruptions  to  one  of  the  critical 
nodes  or  links  in  the  network  can  propagate  through  the  network  and  result  in  degradation  to 
seemingly  distant  nodes  (Huang  et  al.,  2008). 

In  this  study,  we  express  inherent  risk  as  a  density  function  that  describes  the 
probability  of  a  disruption  occurring  at  any  time  during  the  system  development.  We 
concentrate  on  the  Implementation  and  Integration  Phase  as  the  development  stage  where 
disruptions  occur.  Here,  inherent  risk  is  the  probability  of  disruptions  due  to  the 
development  characteristics  of  the  subject  system  (e.  g.,  technology  readiness-level, 
funding,  politics,  etc.).  Risk  due  to  interdependencies,  on  the  other  hand,  is  the  probability 
of  disruptions  during  the  Implementation  Phase  of  a  system  due  to  disruption  in  the  system 
on  which  the  system  of  interest  depends.  This  is  essentially  the  conditional  probability  of  a 
disruption,  given  that  another  system  has  a  disruption. 

This  study  assumes  that  the  inherent  risk  of  a  system  /'  in  satisfying  requirement  r, 
RSys(i,r),  is  solely  a  function  of  its  readiness-level,  m(i,r).  While  a  somewhat  simplified 
definition,  expressing  risk  as  a  function  of  a  system’s  readiness-level  is  logical  since 
readiness  describes  the  necessary  development  of  a  system  to  satisfy  a  given  requirement. 
Therefore,  risk  changes  as  the  readiness-level  of  a  system  increases.  Equation  1 
introduces  a  relationship  between  a  system’s  readiness-level  and  inherent  risk  (probability  of 
disruption). 


(1) 


In  this  relationship,  a,  (with  a  value  between  0  and  1)  is  a  parameter  that  indicates 
the  upper-bound  value  of  risk  for  system  /  (i.e.,  producing  maximum  probability  of  disruption) 
while  Pi  is  a  shape  parameter  that  indicates  how  quickly  risk  changes  as  a  function  of 
readiness-level.  This  formulation  implies  that  risk  is  highest  at  the  early  stages  of 
development  (e.g.,  low  readiness-levels)  and  it  decreases  (at  different  rates,  depending  on 
the  value  of  the  Pi  parameter)  as  development  progresses.  For  instance,  when  a  system  / 
has  a  readiness-level  of  0.0 — it  is  a  brand  new  system — the  probability  of  disruptions  during 
development  will  be  highest,  and  it  will  have  a  value  a-,.  However,  when  the  system  has  a 
readiness-level  of  1 .0,  the  probability  of  disruptions  will  be  0.  System  inherent-risk  is 
implemented  in  the  CEM  by  using  a  uniform  random  distribution  to  select  a  value  between  0 
and  1  at  each  time-step  of  the  Implementation  or  Integration  Phase  and  passing  it  into  a 
binary  channel  to  see  if  the  number  is  smaller  or  greater  than  the  probability  of  disruption 
defined  by  Rsys(iJ)-  This  determines  if  a  disruption  occurs  or  not. 

When  all  systems  are  independent,  identification  of  the  system  with  highest  risk  is 
trivial  (e.g.,  the  system  that,  on  average,  will  contribute  more  to  delays  in  completion  time). 
However,  when  systems  are  interdependent,  systems  that  otherwise  have  a  low  inherent 
risk  can  be  greatly  impacted  by  disturbances  because  of  the  transmission  of  risk  from  other 
systems.  Systems  are  impacted  by  nearest  neighbors  (those  systems  on  which  they  directly 
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depend;  first-order  dependencies)  and  by  systems  that  impact  those  nearest  neighbors 
(higher-order  dependencies). 

The  CEM  models  risk  due  to  interdependencies  in  terms  of  the  dependency  strength 
between  two  given  systems.  Dependency  strength,  S{i,j),  is  an  input  parameter  that  takes 
values  between  0  and  1  and  is  defined  as  the  conditional  probability  (uniform  random 
probability)  that  system  /  has  a  disruption,  given  that  system  j  (on  which  system  /'  depends) 
has  a  disruption.  Risk  due  to  interdependencies  is,  therefore,  a  function  of  the  readiness- 
level  of  the  dependent-upon  system  as  well  as  the  strength  of  that  dependency. 

When  considering  the  development  of  different  system  sets  that  can  provide  a 
desired  SoS  capability,  the  characteristics  of  interdependencies  must  be  considered 
because  they  have  a  large  influence  on  both  capability  and  development  time.  Quantifying 
the  impact  that  such  characteristics  have  on  the  development  process  can  aid  decision¬ 
makers  in  selecting  the  most  promising  alternative.  The  next  section  of  this  paper  presents 
a  proof-of-concept  application  of  the  CEM  to  perform  an  analysis  of  alternatives  study  for 
different  constituent  systems  of  a  development  network  while  comparing  capability  and 
development  time. 

Proof-of-Concept  Application 

The  ABL  program  serves  as  the  proof-of-concept  problem  for  demonstrating  the 
Computational  Exploratory  Model  (CEM),  equipped  with  a  capability  estimate  module,  for 
performance  of  trade-off  analyses  between  capability  and  development  time.  The  CEM 
simulates  the  propagation  of  disruptions  in  the  network  of  component  system 
interdependencies  and  enables  a  trade-off  study  between  the  completion  time  of  the  ABL 
and  its  potential  capabilities  when  different  component  system  alternatives  are  considered. 

The  ABL  is  a  theater  defensive  weapon  concept  that  is  designed  to  destroy  ballistic 
missiles  in  their  boost  phase  within  the  first  two  minutes  of  flight  from  hundreds  of  kilometers 
away  (Davey,  2000).  The  current  ABL,  still  under  development,  consists  of  an  aerial 
platform  (a  modified  Boeing  747-400),  infrared  sensors  for  detecting  the  missile,  two  solid 
state  lasers  for  tracking  the  missile  and  measuring  atmospheric  disturbances,  an  Adaptive 
Optics  System  (AOS)  for  adjusting  for  atmospheric  disturbances,  and  a  Chemical  Oxygen 
Iodine  Laser  (COIL  beam)  for  destroying  the  missile.  Figure  2  presents  these  component 
systems  and  their  layout  in  the  Boeing  B747-400,  as  described  in  (Defense  Industry  Daily, 
2009).  Note  that  the  ABL  program  may  not  be  considered  a  system-of-systems 
operationally,  but  developmental^,  it  has  all  of  the  traits  required  of  an  SoS,  as  described  by 
Maier  (1998).  In  particular,  the  geographic  distribution,  along  with  managerial  and 
operational  independence,  qualifies  the  development  process  of  the  ABL  as  a  system  of 
systems.  Development  of  the  ABL  team  is  undertaken  by  three  companies,  who  operate 
and  manufacture  their  respective  pieces  of  the  ABL  across  the  country.  The  Beam 
Control/Fire  Control  (BC/FC)  system  is  designed  by  Lockheed  Martin,  the  COIL  beam  is 
designed  by  Northrop  Grumman,  and  the  modifications  to  the  aircraft  and  integration  of 
systems  are  performed  by  Boeing.  In  addition,  each  company  has  been  able  to,  at  least 
partially,  test  their  portions  of  the  ABL  separately  (Davey,  2000),  indicating  some  degree  of 
operational  independence. 
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Figure  2.  Airborne  Laser  Component  Systems 

(Defense  Industry  Daily,  2009) 

The  ABL  operates  as  follows:  first,  several  onboard  Infrared  Search  and  Track 
(IRST)  sensors  detect  the  heat  radiated  by  the  exhaust  of  the  missile.  Next,  a  solid  state 
laser  (the  Track  Illuminator)  tracks  one  or  more  missiles,  determines  an  aim  point,  and 
passes  the  information  to  the  main  ABL  computers.  The  other  solid  state  laser  (the  Beacon 
Illuminator)  measures  disturbances  in  the  atmosphere  so  that  they  may  be  corrected  by  the 
AOS  in  order  to  accurately  focus  the  main  laser  on  the  missile.  This  sequence  adjusts  the 
focus  of  the  COIL  beam  and,  together,  is  known  as  the  Beam  Control/Fire  Control  (BC/FC) 
system.  Finally,  the  COIL  beam — a  dual  line,  multi-module  laser — is  focused  onto  the 
missile  through  a  large  turret  on  the  nose  of  the  vehicle  until  it  compromises  the  structural 
integrity  of  the  missile. 

Several  assumptions  and  simplifications  are  necessary  to  facilitate  the  proof-of- 
concept  study.  While  the  requirements  of  the  ABL  are  comprised  of  several 
components/tasks — detect,  track,  aim  and  adjust  laser  beam,  and  destroy  missile — here 
they  are  grouped  into  a  single  requirement.  Additionally,  the  component  systems  of  the  ABL 
are  grouped  into  four  core  systems:  the  aircraft  system,  the  detection  and  tracking  (D&T) 
system,  the  AOS,  and  the  COIL  beam  system.  Development  of  these  four  systems  and  their 
integration  results  in  the  ABL  capability  of  detecting,  tracking,  and  destroying  theater  ballistic 
missiles  in  their  boost  phase. 

ABL  Capability 

The  capability  of  a  system  is  embodied  by  the  quality  with  which  it  performs  required 
functions.  The  required  capability  of  the  ABL,  as  described  by  Barton  et  al.  (2004),  is  to 
disable  ballistic  missiles  in  their  boost  phase.  Depending  on  the  type  of  threat  missiles, 
operating  environment,  and  other  operational  variables,  many  metrics  exist  for  describing 
the  capability  of  the  ABL  system.  In  this  work,  we  assume  that  the  ABL  capability  of  interest 
is  its  ability  to  disable  threats  from  a  range  of  600  km.  Tests  and  studies  of  the  ABL  have 
shown  that  600  km  is  a  reasonable  performance  goal  (Barton  et  al.,  2004).  Hence,  the 
achievable  capability  level  of  the  ABL  will  be  measured  against  this  baseline  value  of 
engagement  range. 

Three  functions  are  necessary  on  the  ABL:  detect  and  track  the  missile,  engage  the 
missile,  and  disable  the  missile.  As  previously  mentioned,  we  assume  that  four  constituent 
systems  comprise  the  ABL  system  and  perform  the  three  functions.  The  contributions  of 
each  system  to  the  execution  of  each  function  are  presented  in  Figure  3. 
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Figure  3.  Assumed  Capability  Composition  of  ABL 

The  capability  of  the  ABL  is,  therefore,  a  function  of  the  performance  levels  of  each 
of  its  constituent  systems  (Table  2). 

Table  2.  Performance  Goals  of  ABL  Systems 


Constituent  System 

Performance  Metric 

Performance  Level  (units) 

Detection  &  T racking 

Detection  time,  Td 

10  (sec) 

Aircraft 

Payload  capacity 

250,000  (lbs) 

COIL  beam 

Beam  power,  P 

5  (MW) 

Adaptive  Optics 

Beam  quality,  bq 

1.2  (n/a) 

The  detection  time,  Td,  is  the  time  that  the  D&T  system  requires  to  acquire  a  target 
and  generate  a  track.  This  is  an  important  performance  parameter  because  it  will  dictate  the 
time  available  to  the  laser  to  engage  and  disable  the  target  during  the  boost  phase.  Based 
on  the  report  by  Barton  et  al.  (2004),  an  acceptable  dwell  time  (the  amount  of  time  that  the 
laser  must  deliver  its  energy)  for  a  liquid-propelled  missile  is  on  the  order  of  4  to  5  seconds. 
This  means  that  for  a  given  raid  size,  the  D&T  system  has  a  limited  time  to  acquire  the 
target  and  generate  tracks.  We  assume  that  in  order  for  the  ABL  to  disable  up  to  12 
simultaneously  launched  liquid  fuel  missiles  (with  a  boost  phase  of  170  seconds),  the  ideal 
detection  time  is  10  seconds  (based  on  a  dwell  time,  te,  of  4.2  seconds;  Equation  2). 


irp-esffifjis  ,  *. 
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The  COIL  beam  is  the  centerpiece  of  the  ABL  system.  The  beam  power,  P, 
determines  the  amount  of  energy  that  will  be  delivered  to  the  missile.  Again,  based  on  the 
extensive  report  by  Barton  et  al.  (2004),  a  reasonable  power  performance  for  the  COIL 
beam  is  around  5  MW.  The  capability  of  the  ABL  will  be  a  function  of  this  performance 
parameter  as  well  as  the  performance  of  the  other  constituent  systems. 


The  aircraft  hosts  the  other  constituent  systems  of  the  ABL  and  provides  the 
necessary  mobility  characteristics  of  this  weapon.  However,  we  assume  that  from  a 
capability  point  of  view  (to  disable  a  missile  from  600  km),  it  can  fulfill  the  necessary 
requirements  to  host  the  constituent  systems  of  the  ABL  and  is  thus  not  a  part  of  the 
capability  trade  space.  Note  that  this  is  a  simplifying  assumption  in  this  study  but  one  that 
can  be  included  in  more  detailed  studies. 


Finally,  atmospheric  disturbance  must  be  accounted  for,  since  it  plays  a  significant 
role  on  the  laser  performance.  Development  of  the  ABL  system  includes  the  development  of 
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an  advanced  Adaptive  Optics  System  that  can  account  for  the  atmospheric  disturbances 
and  increase  the  energy  delivered  to  the  missile.  The  performance  of  these  optics  is 
typically  described  by  the  Strehl  ratio.  The  Strehl  ratio  is  a  measure  of  the  quality  of  optics 
that  compares  the  peak  intensity  at  the  detection  point  with  a  theoretical  maximum  intensity. 
While  various  factors  contributed  to  the  quantification  of  the  Strehl  ratio,  Barton  et  al.  (2004) 
provide  the  simplified  description: 


where  bq  is  the  beam  quality  diffraction  limit  and  can  be  used  as  a  performance  benchmark 
for  adaptive  optics.  Barton  et  al.  (2004)  state  that  a  beam  quality  value  of  1 .2  represents  a 
reasonable  goal. 

The  amount  of  energy  required  to  disable  a  missile  varies  according  to  the  missile 
construction  and  the  type  of  fuel  it  utilizes  (fuel  tanks  are  the  most  vulnerable  part  of  the 
missile).  Barton  et  al.  (2004)  offer  a  simplified  relationship  between  the  performance 
parameters  of  its  constituent  systems  and  the  capability  of  the  ABL  to  disable  a  missile  from 
a  distance  R.  In  this  relationship,  the  force  required  to  disable  a  missile,  Fc,  is  expressed  as 
follows: 


(4) 


where  D  and  A  are  the  diameter  and  wavelength  of  the  COIL  beam,  respectively;  R  is  the 
slant  range  (e.g.,  the  distance  between  the  ABL  and  the  target  missile);  P  is  the  COIL  beam 
power  in  Watts;  te  is  the  laser  dwell  time  (e.g.,  the  time  that  the  laser  delivers  its  energy  to 
the  target);  and  SR  is  the  Strehl  ratio  of  the  Adaptive  Optics  System.  Solving  this  relationship 
for  the  slant  range,  R,  describes  the  capability  of  the  ABL  as  a  function  of  the  performance 
parameters  of  its  constituent  systems. 


(5) 


The  capability  contributed  by  the  COIL  beam  is  represented  by  the  COIL  beam 
power,  P,  (and  fixed  values  of  D  =  1 .5  m  and  A  =  1 .315  pm  );  the  capability  contributed  by 
the  AOS  is  represented  by  the  SR  value;  and  the  capability  contributed  by  the  D&T  system  is 
represented  by  the  available  dwell  time,  te.  We  assume  that  the  capability  of  the  ABL  will  be 
measured  in  terms  of  its  ability  to  disable  a  liquid-fueled  ICBM,  which  requires  a  force  of  32 
MJ,  Fc  =  32  MJ/m2  (Barton  et  al.,  2004).  The  capability  of  the  ABL  will  be  computed  by  using 
this  relationship  for  different  combinations  of  constituent  systems  that  can  offer  various 
levels  of  system-specific  performance  and  will  be  compared  to  their  estimated  development. 

ABL  Development 

The  Air  Force  and  the  Missile  Defense  Agency  (MDA)  have  been  experimenting  with 
the  simultaneous  development,  testing,  and  integration  of  the  component  systems  of  the 
ABL.  Because  of  this,  development  of  these  systems  is  interdependent.  For  instance,  the 
aircraft  developer  needs  the  stability  requirements  and  dimensional  specifications  of  the 
Adaptive  Optics  System  and  the  COIL  beam  system  to  determine  the  proper  mountings  and 
fuselage  dimensions  of  the  aircraft;  or,  development  of  the  aircraft  requires  knowledge  of  the 
heat  dissipated  by  the  COIL  beam  to  determine  the  amount  of  heat  protection  to  include  in 
the  aircraft  airframe  and/or  subsystems.  Depending  on  the  performance  of  the  COIL 
beam — i.e.,  its  maximum  power  output — the  adaptive  optics  must  provide  a  certain  level  of 
performance  in  order  to  deliver  the  required  amount  of  energy  to  the  target.  Similarly, 
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depending  on  the  capability  of  the  D&T  system,  the  adaptive  optics  must  be  able  to 
effectively  compensate  for  the  atmospheric  disturbances  of  the  detection  range. 
Development  of  the  AOS  is,  therefore,  dependent  on  the  development  of  the  COIL  beam 
and  the  D&T  system.  A  representation  of  the  interdependencies  in  this  example  problem 
and  its  layered  network  structure  is  presented  by  Figure  4. 


Figure  4.  Assumed  System  Interdependencies  in  ABL  Example 

While  more  interdependencies  may  be  present  in  the  development  of  the  ABL 
systems,  for  the  purpose  of  this  demonstration,  we  assume  that  the  topology  presented  in 
Figure  4  represents  the  development  interdependencies  of  the  ABL  system  and  remains 
fixed  during  the  analysis  of  alternatives.  The  goal  is  to  present  a  sample  utilization  of  the 
CEM  to  perform  analysis  of  alternatives  and  capability  and  development  risk  trade-off.  The 
CEM  will  utilize  these  interdependency  characteristics  and  other  necessary  parameters  to 
estimate  the  development  time  of  the  ABL  when  alternative  constituent  systems  (e.g., 
systems  with  varying  levels  of  capability)  are  considered. 

Results 

For  the  proof  of  concept  application  presented  here,  the  desired  capability  is  the 
ability  to  engage  and  disable  missiles  from  a  range  of  600  km.  This  capability  is  a  function  of 
the  constituent,  interdependent  systems.  Here,  we  assume  that  the  designer  has  the  option 
to  select  different  constituent  systems  to  satisfy  this  ABL  requirement.  The  Boeing  B747- 
400  is  currently  being  used  as  the  aerial  platform  that  hosts  the  ABL  system.  The  MDA 
stated  in  a  2007  report  that  an  alternative  to  the  current  ABL  platform  is  the  utilization  of 
Unmanned  Aerial  Vehicles  (UAVs),  which  can  offer  longer  endurance  and  eliminate  the  risk 
to  crew  members.  Similarly,  Davey  (2000)  reports  that  alternate  systems  to  the  currently 
used  detection  and  tracking  system  could  be  considered  to  partially  fulfill  the  ABL 
requirement  (e.g.,  UAV  or  Space  Tracking  and  Surveillance  System  [STSS]).  Additionally, 
Barton  et  al.  (2004)  indicated  that  the  ideal  performance  of  the  Adaptive  Optics  System  and 
the  COIL  beam  is  still  questionable,  and  sub-optimal  “solutions”  will  be  utilized  following  a 
spiral  development  strategy  that  will  enable  incremental  improvement  of  these  systems’ 
capabilities. 
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Three  alternative  aerial  platforms  and  three  detection  and  tracking  systems  are 
considered  to  fulfill  the  ABL  requirement,  while  three  levels  of  performance  of  the  AOS  and 
the  COIL  beam  with  different  levels  of  initial  readiness-level  are  considered.  Table  3 
presents  these  assumed  values  for  alternatives  for  the  aircraft  system. 


Table  3.  Assumed  Values  for  Alternative  Systems  for  Aerial  Platform 


Aircraft 

Alternative 

Max  Payload 

[lbs] 

TRL 

Initial  Readiness-Level 
[m°(i,r)\ 

Implementation  Pace 
[Pimp{i)] 

new  aircraft 

TBD 

5 

0.56 

0.04 

KC-135A 

105,821 

6 

0.67 

0.04 

B747-400 

248,000 

8 

0.89 

0.04 

All  alternatives  are  assumed  to  have  an  implementation  pace  of  0.04;  this  means 
that  at  every  time-step  during  the  OEM  simulation,  the  completeness-level  increases  by  an 
increment  of  0.04,  until  a  completeness-level  of  1.0  is  reached.  The  Boeing  NKC-135A  is 
included  here  as  an  alternate  aerial  host  platform  because  it  was  the  primary  aircraft  in  the 
Airborne  Laser  Laboratory  (ALL)  — a  precursor  to  today’s  Airborne  Laser  program — during 
the  1980s  (Duffner,  1997).  The  purpose  of  this  program  was  to  perform  tests  and  determine 
whether  or  not  a  laser  mounted  on  an  aircraft  could  actually  shoot  down  an  airborne  target. 
The  Boeing  747-400  is  the  aircraft  that  currently  hosts  the  constituent  systems  of  the  ABL 
and  has  a  payload  capacity  of  248,000  lbs  ( Jane’s  All  the  World’s  Aircraft,  2010).  A  GAO 
report  (2002)  stated  that  the  present  laser  with  six  modules  weighs  180,000  lbs  and  the 
laser  design  calls  for  a  laser  with  14  modules;  while  the  actual  power  output  of  the  laser  is 
not  known,  we  assume  a  linear  relationship  between  the  weight  of  the  laser  and  its  power 
output,  and,  therefore,  a  larger  payload  capacity  is  required  for  the  aircraft  to  host  the  sub¬ 
systems  of  the  COIL  beam.  The  new  aircraft  alternative  is  assumed  to  provide  this  required 
payload  capability. 

Furthermore,  because  modifications  are  necessary  to  host  the  other  component 
systems  of  the  ABL,  we  assume  that  the  KC-135A,  the  B747-400,  and  the  new  aircraft  have 
a  TRL  of  6,  8,  and  5,  respectively.  We  utilize  the  TRL  as  an  indicator  of  the  risk  associated 
with  the  development  of  a  given  system;  the  approach  followed  here  normalizes  the  TRL 
value  (by  dividing  by  the  maximum  possible  TRL,  9)  and  uses  this  value  as  the  initial 
readiness-level  of  the  system  (m°).  The  new  aircraft  alternative  has  the  lowest  TRL  because 
it  is  a  brand  new  system;  however,  it  does  not  have  a  TRL  of  0  because  we  assume  that 
existing  technologies  can  be  utilized  to  meet  its  requirements. 

The  options  to  the  designer  for  the  detection  and  tracking  system  of  the  ABL  are  to 
design  a  brand  new  system  or  use  legacy  systems  like  the  Space  Tracking  and  Surveillance 
System  (STSS)  or  UAVs.  Table  4  presents  the  alternate  systems  and  assumed  capabilities 
along  with  their  initial  readiness-levels. 


Table  4. Assumed  Values  for  Alternative  Systems  for  Detection  System 


Detection 

Alternative 

Detection 
Time  [sec] 

TRL 

Level 

Initial  Readiness- 
Level  [m°(i,r)] 

Implementation 

Pace  [Pimp(i)] 

New  System 

10 

6 

0.67 

0.04 

UAV 

11 

8 

0.89 

0.04 

STSS 

12 

9 

1.00 

0.04 
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One  option  that  the  MDA  has  considered  for  the  early  detection  and  targeting  of 
missiles  is  the  utilization  of  UAVs  (Buttler,  2009).  However,  because  current  concepts  of 
operations  involve  the  UAV  accepting  a  cue  from  satellites  about  the  threat  missile,  we 
assume  that  the  detection  time  for  such  a  system  is  of  1 1  seconds.  Recall  that  detection 
time  impacts  the  available  laser  dwell  time  (e.g.,  longer  detection  time  reduces  the  available 
time  to  disable  the  missile  during  the  boost  phase).  Furthermore,  because  UAVs  are 
currently  used  to  perform  reconnaissance  missions,  we  assume  that  utilizing  UAVs  for 
detection  and  tracking  has  a  TRL  level  of  8.  Another  option  for  detecting  and  tracking  the 
missile  is  the  use  of  the  Space  Tracking  and  Surveillance  System  (STSS).  As  of  2003,  the 
MDA  has  decided  to  fund  the  design  but  not  the  production  of  a  competitive  sensor  for  use 
aboard  the  satellites  (Smith,  2003).  We  assume  that  the  STSS  has  a  TRL  level  of  9  and  can 
achieve  a  detection  time  of  12  seconds  if  it  is  used  as  the  detection  and  tracking  system  of 
the  ABL.  Finally,  we  consider  the  development  of  a  new  system  to  provide  the  D&T 
capability  for  the  ABL  system.  Based  on  the  GAO  report  (2002),  the  D&T  system  under 
development  has  a  TRL  level  of  6.  Because  this  is  a  custom  system  designed  specifically  for 
use  in  the  ABL  system,  we  assume  that  it  can  achieve  a  detection  time  of  10  seconds,  which 
would  enable  the  detection  of  up  to  12  simultaneously  launched  missiles  before  the  end  of 
the  boost  phase,  assuming  a  170-second  boost  phase  and  a  dwell  time  of  4.2  seconds. 

While  alternative  systems  for  the  aerial  platform  and  the  D&T  system  exist,  the  COIL 
beam  and  the  Adaptive  Optics  System  are  new  technologies  for  which  alternatives  do  not 
exist.  Because  the  level  of  performance  of  these  systems  is  still  uncertain,  we  assume  that 
different  levels  of  beam  quality  and  power  output  for  the  AOS  and  COIL  beam,  respectively, 
can  be  achieved  given  the  different  TRL  levels.  Table  5  and  Table  6  present  these 
assumed  values. 


Table  5.  Assumed  Values  for  Alternative  Systems  for  Adaptive  Optics  System 


Detection 

Alternative 

Beam  Quality 
Diffraction  Limited 

TRL  Level 

Initial  Readiness-Level 
[m°m 

Implementation 
Pace  [p/mp(/')] 

Alternative  1 

1.2 

2 

0.22 

0.02 

Alternative  2 

1.3 

3 

0.33 

0.02 

Alternative  3 

1.4 

5 

0.56 

0.02 

Table  6. 

Assumed  Values  for  Alternative  Systems  for  COIL  beam  System 

COIL  Beam 
Alternative 

Power 

[MW] 

TRL 

Level 

Initial  Readiness-Level 
[m°m 

Implementation 
Pace  [Pimp(i)] 

Alternative  1 

3 

4 

0.44 

0.03 

Alternative  2 

4 

3 

0.33 

0.03 

Alternative  3 

5 

1 

0.11 

0.03 

The  GAO-02-631  report  (2002)  provides  the  TRLs  for  Alternative  1  for  both  the  AO 
and  the  COIL  beam  systems,  and  assumed  TRL  and  capability  values  are  used  for  the  other 
alternatives,  as  well  as  implementation  paces.  The  systems  engineer  would  like  to  know 
which  combination  of  constituent  systems  results  in  a  (ABL)  system  with  lowest  estimated 
completion  time  and  provides  the  largest  capability  potential.  We  assume  that  all 
alternatives  have  a  maximum  probability  of  disruption  of  0.2  (a,  =  0.2),  which  decreases  as 
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the  completeness-level  of  a  system  increases.  This  implies  that  alternatives  with  a  larger 
initial  readiness-level  will  have  a  smaller  probability  of  disruption  than  systems  with  a  smaller 
initial  readiness-level. 

In  the  present  study,  the  interdependency  strengths  between  systems  are  varied  for 
each  potential  ABL  architecture,  based  on  the  initial  readiness-level  of  the  candidate 
constituent  system.  We  assume  that  the  initial  readiness-level  of  a  given  system  indicates 
the  interdependency  strength  between  that  system  and  all  the  systems  that  depend  on  it 
and  that  the  interdependency  strength  is  the  complement  of  the  initial  readiness-level.  For 
instance,  if  one  of  the  alternatives  for  the  COIL  beam  has  an  initial  readiness-level  of  0.33, 
then  the  strength  of  the  dependency  of  the  aircraft  system  on  the  COIL  beam  system  is  0.77 
(1-0.33). 

Based  on  the  alternative  systems  in  Table  3,  Table  4,  Table  5,  and  Table  6,  there  are 
81  possible  combinations  of  D&T,  aircraft,  COIL  beam,  and  Adaptive  Optics  Systems  that 
could  satisfy  the  requirement  of  the  ABL,  albeit  at  a  different  capability  level.  For  the 
purpose  of  this  study,  these  describe  the  design  space  for  the  analysis  of  alternatives.  The 
goal  is  to  quantify  the  trade-off  between  the  ABL  capability  (in  terms  of  the  engagement 
range)  and  the  estimated  development  time.  To  simplify,  we  assume  that  the 
interdependencies  between  the  systems  will  not  change  in  the  scenarios  where  the 
alternative  systems  are  utilized  (i.e.,  the  system  interdependencies  presented  in  Figure  4  will 
be  invariant). 

CEM  simulates  the  development  process  and  estimates  the  completion  time  of  the  entire 
program  and  uses  system-specific  capabilities  to  compute  the  ABL  capability.  Recall  that 
the  initial  readiness  level  determines  the  maximum  risk  of  the  initial  stages  of  the 
development  process.  The  estimated  completion  time,  therefore,  reflects  the  impact  that 
risk  (both  inherent  and  due  to  interdependencies)  has  on  the  completion  time  of  the  ABL 
program  for  the  different  alternative  systems,  their  combinations,  and  implementation 
strategies.  Figure  5  presents  the  expected  completion  time  of  the  ABL  project,  as  estimated 
by  the  computational  model  and  the  potential  capability  for  the  81  combinations  of 
alternative  systems. 
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Figure  5.  Tradeoff  Between  Expected  Completion  Time  and  Potential  Capability  of 

ABL 

The  seven  solutions  called  out  in  Figure  5  represent  the  seven  combinations  of  these 
alternative  systems  that  yield  promising  combinations  of  capability  and  expected  completion 
time.  They  are  the  non-dominated  solutions  of  this  trade-off  study  and,  as  such,  define  a 
Pareto  Frontier.  Essentially,  by  choosing  any  of  these  seven  solutions,  it  is  impossible  to 
improve  the  expected  completion  time  without  giving  up  capability.  Table  7  lists  the  systems 
that  comprise  each  of  these  seven  solutions,  the  resulting  potential  capability,  and  expected 
completion  time. 


Table  7.  Description  of  Non-Dominated  Solutions 


Solution 

D&T  System 

Aircraft 

System 

COIL  Beam 
System 

AO  System 

ABL  Capability 

[slant  range, 
km] 

Expected 

Completion 

Time 

[time  units] 

1 

STSS 

new  system 

Alternative-1 

Alternative-3 

285 

152 

2 

STSS 

new  system 

Alternative-1 

Alternative-2 

307 

153 

3 

UAV 

new  system 

Alternative-1 

Alternative-2 

371 

157 

4 

UAV 

new  system 

Alternative-1 

Alternative-1 

402 

160 

5 

new  system 

new  system 

Alternative-1 

Alternative-1 

461 

170 

6 

new  system 

new  system 

Alternative-2 

Alternative-1 

533 

185 

7 

new  system 

new  system 

Alternative-3 

Alternative-1 

596 

215 

As  expected,  developing  brand  new  systems  for  the  D&T  and  aircraft  system, 
combined  with  high  capability  COIL  beam  and  AO  system  alternatives,  produces  the 
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maximum  possible  capability  (assuming  that  requirements  will  not  change)  but  also  the 
highest  development  time.  Systems  that  provide  the  highest  level  of  performance  also  have 
the  lowest  initial  readiness-levels,  which,  in  turn,  means  high  development  risk.  Conversely, 
utilizing  legacy  systems  with  a  relatively  high  readiness-level  (e.g.,  STSS,  alternative-1  for 
the  COIL  beam  system,  and  alternative-3  for  the  AOS)  results  in  the  shortest  development 
time  but  also  the  lowest  capability-level.  The  model  results,  at  these  extremes,  are  verified 
with  our  intuition. 

A  new  aircraft  system  is  always  preferred.  Recall  that  the  aircraft  does  not  impact  the 
ABL  capability  here  but  contributes  to  the  development  time.  Furthermore,  for  the  first  five 
non-dominated  solutions,  the  COIL  beam  system  that  has  the  lowest  capability  (e.g.,  3  MW 
of  power  output)  is  selected.  This  means  that  the  expected  development  time  to  be  incurred 
to  achieve  higher  power  output  is  not  worth  the  increase  in  the  ABL  capability  (for  the 
assumed  risk  values  used  here).  Conversely,  the  Adaptive  Optics  System  selected  for  the 
last  four  solutions  (4-7)  is  the  alternative  that  provides  the  highest  capability.  This  means 
that  the  expected  higher  development  time  of  this  system  justifies  the  potential  capability 
that  it  can  provide  to  the  ABL.  These  results  align  with  the  observations  of  Barton  et  al. 
(2004),  who  showed  in  their  sensitivity  studies  of  the  ABL  capabilities  that  improvements  in 
the  COIL  beam  power  output  are  not  as  critical  as  the  ability  of  the  Adaptive  Optics  System 
to  correct  for  the  atmospheric  disruptions  and  deliver  the  required  energy  to  the  target. 

Although  the  capability  and  initial  readiness-level  values  of  the  candidate  systems  in 
this  study  were  assumed,  the  trade-off  study  represents  a  very  real  decision-making 
situation  for  system  engineers  doing  AOA  in  pre-milestone  B  portions  of  the  acquisition 
process.  The  approach  could  be  improved  by  using  physics-based  modeling  tools  for 
technical  capacity  and  initial  readiness-level  estimation,  as  well  as  process  modeling  for  the 
impact  of  disruptions  under  different  system  implementation  strategies.  The  CEM  enables 
this  type  of  investigation  by  considering  the  relatively  explicit  inherent  development  risk  of 
component  systems  as  well  as  the  implicit  risk  due  to  system  interdependencies. 

Conclusions 

The  development  of  complex  systems  (and  SoS)  is  beset  by  risk.  Risk  analyses  of 
individual  systems  can  explain  the  threats  and  opportunities  of  systems  but  do  not  capture 
the  impact  that  disruptions  to  individual  systems  have  at  the  enterprise  level,  where  multiple 
systems — explicitly  or  implicitly  interdependent — collaborate  to  achieve  various  capabilities. 
The  presence  of  interdependencies  in  layered  networks  of  development  systems  often  result 
in  increased  risk  and  higher  order  disruptions  that  are  not  always  visible  or  predictable.  The 
network  structure  behind  the  collaboration  can  contribute  both  negatively  and  positively  to 
the  successful  achievement  of  SoS  capabilities  and,  even  earlier,  to  the  developmental 
success.  Collaboration  via  interdependence  may  increase  capability  potentials,  but  it  also 
contains  concealed  risk  in  the  development  and  acquisition  phases. 

This  paper  considered  the  Airborne  Laser  system  under  development  by  the  Missile 
Defense  Agency  to  present  the  CEM,  its  parameters,  and  example  trade-off  studies 
between  estimated  completion  time  of  the  program  and  its  potential  capability.  Results  of 
the  analysis  of  this  simplified  system  revealed  that  a  Pareto  Frontier  exists  when  the 
completion  time  of  a  project  is  compared  to  the  potential  capability  that  it  can  provide.  In 
this  example,  only  seven  of  the  81  combinations  of  alternative  systems  for  the  aircraft  and 
detection  and  tracking  systems  were  non-dominated  solutions.  The  highest  capability  (and 
highest  completion  time)  was  achieved  when  all  component  systems  were  developed  from 
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scratch  and,  conversely,  the  lowest  capability  (and  lowest  completion  time)  was  a  result  of 
utilizing  mature  legacy  systems  that  require  minimal  modifications. 

The  Computational  Exploratory  Model  presented  here  is  an  ongoing  research  effort 
that  aims  to  provide  a  framework  for  the  aggregation  of  the  system-specific  risk  to  the 
enterprise  level.  The  extensions  to  the  model  presented  here  via  a  proof-of-concept 
application  point  to  the  ability  of  such  a  framework  to  quantitatively  perform  analysis  of 
alternatives  and  enable  knowledge-based  acquisition.  It  is  our  goal  to  improve/facilitate  the 
decision-making  process  of  systems  engineers  and  system  integration  by  providing  the 
means  to  model  risk  in  the  system  development  process  and  quantify  the  cascading  effect 
of  risk  for  families  of  systems,  or  SoS,  as  well  as  enable  quantitative  analysis  of  alternatives. 
Analytical  models  in  pursuit  of  the  same  goals  are  also  under  development;  one  version  of 
an  analytical  approach  was  presented  at  the  2010  Annual  Symposium. 
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Development  of  SoS  is  complex 

-  Numerous  interdependencies 

-  Changing  over  time 

SoS  capability  comprised  of 
system  capabilities 

-  Interdependent  system 
requirements 

-  Legacy  systems 

Goal:  make  the  AoA  smarter  in 
pre-acquisition 

-  Potential  capability  vs.  expected 
development 

A  high-level  approach  can  aid  in 
the  early  development  stages  and 
requirement  definition  and 
allocation 
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Research  Questions 

Given  a  network  of  systems 

-  How  do  system-specific  (node)  characteristics  impact  the 
successful  development  of  SoS  capability? 

-  How  do  system  interdependencies  impact  the 
development  process? 

•  How  do  disruptions  propagate  in  complex  networks  of 
interdependent  systems? 

•  How  can  we  quantify  the  cascading  effects  of  development  risk? 

•  Focus  of  previous  year  research 


What  is  the  tradeoff  between  SoS  capability  and 
expected  development  time? 

-  Key  tradeoff  in  analysis  of  alternatives  (AoA) 

-  Focus  of  this  year’s  work 
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Methods  of  Approach 


Simulation  Approach 

-  Developing  Computational  Exploratory 
Model  (CEM) 

-  Discrete-event,  stochastic  simulation 
based  on  steps  in  DoD  SoS  SE  Guide 

-  First-order  modeling  of  capability 


Analytical  Approach 

-  Based  on  probability  and  network  theory 

-  Analysis  of  expected  delay  propagation 
for  given  SoS  network  configurations 
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Current  Research  Efforts 


Analysis  of  alternatives  in  the 
context  of 

-  Development  time 

-  Capability  level 

First-order  capability  estimation 
model 

Capability  /  development  time 
tradeoffs  for  alternative 
compositions  of  Airborne  Laser 
system 

-  Categories  of  components 
comprise  the  capability 

-  Proof  of  concept  application 
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Development  Model  (CEM) 


Discrete-event,  stochastic 
simulation 

•  Disruption  occurrence  and 
propagation 

System  risk  (R  )  as  a  function 
of  system  readiness-level  ( m ) 

-  Similar  to  TRL  metric  and  SRL 
metric  proposed  by  Sauser  et  al. 

Impact  of  disruptions  a  function 
of 

-  Network  topology  and  strength 
of  system  interdependencies 
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Capability  Modeling 


Assume  desired  ABL  capability  to 
“disable  threat  from  600  km  (slant 
range)” 

-  Categories  of  systems  and 
requirements  create  different 
capability  levels 

Identify  functions  that  comprise 
capability 

Identify  systems  that  perform  each 
function 

First-order  quantification  of 
capability 

-  Aircraft  system  indirectly 
considered  (host  of  other 
systems) 
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Capability  Contributors 

Detection  and  tracking  system 


disturbances  to  deliver  maximum 
laser  power  to  target 

-  Capability  contribution:  beam  quality 
diffraction  limited,  b  ,  that  increases 
Strehl  ratio,  SR 

COIL  beam  power 

-  Laser  power  to  disable  a  liquid  fuel 
ICBM 

-  32  MJ/m2  required  (Fc) 
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ABL  Capability  Space 


P=5  MW 


P=4  MW 


ABL  capability  =  f(beam  quality,  laser  power,  available  dwell-time) 
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Analysis  of  Alternatives  Results 

•  81  possible  solutions 

•  Three  alternatives  for  each  constituent  system 

•  Non-dominated  solutions  result  in  a  Pareto  frontier 


•  Clear  tradeoff  between 
capability  and  expected 
development  time 

-  Higher  capability  requires 
higher  development  time 
(result  of  non-mature 
technology) 

•  Seven  solutions  identified 
here 

-  Combination  of  new  and 
existing  systems  (high  and 
low  capability) 
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Observations 


No  single  optimal  solution 

-  Tradeoff  between  capability  and 
development  time 


Non-dominated  solutions  are 
comprised  of  legacy  and  new 
systems 

-  Development  model  captures 
higher  order  impact  of 
interdependencies 


Solution 

D&T 

System 

Aircraft 

System 

COIL  beam 
System 

AO  System 

ABL  Capability 
[slant  range,  km] 

Expected 
Completion  Time 
[time  units] 

1 

STSS 

new  system 

Alternative- 1 

Alternative-3 

285 

152 

2 

STSS 

new  system 

Alternative- 1 

Alternative-2 

307 

153 

3 

UAV 

new  system 

Alternative- 1 

Alternative-2 

371 

157 

4 

UAV 

new  system 

Alternative- 1 

Alternative- 1 

402 

160 

5 

new  system 

new  system 

Alternative- 1 

Alternative- 1 

461 

170 

6 

new  system 

new  system 

Alternative-2 

Alternative- 1 

533 

185 

7 

new  system 

new  system 

Alternative-3 

Alternative- 1 

596 

215 

Purdue 

UNIVERSITY 


^  J 


School  of  Aeronautics  and  Astronautics 


Conclusions 

CEM  and  capability  modeling  enables 
analysis  of  alternatives  early  in  development 
process 

-  CEM  captures  cascading  effect  of  developmental 
disruptions 

•  Enabling  enhanced  selection  of  constituent  systems  and 
requirements 

Analytical  tools  early  in  acquisition  and 
development  phase  enhance  decision-making 

-  Build  intuition  and  guide  acquisition  efforts 
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Future  Work 

Analytical  model  for  measuring  system  development 
performance 

-  Indicators  of  good  network  structure 

-  Identification  of  features  that  can  lead  to  problems  or 
success 

Requirement  evolution  is  at  root  of  most  development 
issues 

-  Want  more  /  better  capability 

-  Get  schedule  and  cost  overruns 

Continue  development  of  a  capability  module  for  CEM 

-  Analysis  of  impact  of  requirement  dependencies  on  both 
development  and  capability 

-  Can  we  “design”  a  controller  for  requirement  evolution? 

•  Ability  to  measure  impact  of  requirement  evolution  on  system 
(and  SoS)  development 
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Markov  Perspective  on  Network 
Interdependencies 


Aggregation  of  system-specific 
disruptions  to  generate  network-level 
performance  metric 

-  Focus  on  cascading  effect  of  disruptions 

-  Identify  network  characteristics  that 
increase  probability  of  project  success 

Proposed  approach  gives  ability  to 

-  Rank  constituent  systems  based  on 
criticality/vulnerability  during 
development 

-  A  network-level  metric  enables 
comparison  of  networks  (that  can  vary 
with  time) 


•  • 
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Network-Level  Metric 


Compute  expected  accumulated  delay 

-  Measure  of  network  performance 

•  Measure  of  system  criticality  /  vulnerability 
when  contributions  from  each  system  are 
ranked 

Compute  variation  about  the 
expectation 

-  Measure  of  the  risk  associated  with  the 
estimated  network  performance 

tj(n  +  \)=  A<j(n)  subject  to  £(0)  =  b 
F(n\Xj(0))=cA») 

E\F{p\xj{Q%  =  YdncAnb 

n=  1 
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Thank  You 
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Back-Up  Slides 
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Contributors  to  Capability:  Detection  &  Tracking 

•  Capability  assumptions 

-  170  seconds  of  boost-time  (engagement  window) 

-  Desired  raid  size  of  12  missiles:  determines  required  dwell- 
time 

•  Ideal  detection  time  is  10  seconds;  allows  interception  of  12  missiles 

•  Development  assumptions 

-  Normalized  TRL  indicates  initial  readiness-level 

•  Determines  probability  of  disruptions  during  development 


Detection  Alternative 

Detection  time 
[sec] 

TRL 

Level 

Initial  Readiness-Level 

[m°(i,r)\ 

New  System 

10 

6 

0.67 

UAV 

11 

8 

0.89 

STSS 

12 

9 

1.00 
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Contributors  to  Capability:  Adaptive  Optics 

•  Capability  assumptions 

-  Only  a  function  of  the  beam  quality  diffraction  limit,  b 

-  Ideal  beam  quality  diffraction  limited  is  1 .2 

•  Development  assumptions 

-  Normalized  TRL  indicates  initial  readiness-level 

•  Determines  probability  of  disruptions  during  development 


Detection 

Alternative 

Beam  Quality 
Diffraction  Limited 

TRL  Level 

Initial  Readiness-Level 
[m'Ur)] 

Alternative  1 

1.2 

2 

0.22 

Alternative  2 

1.3 

3 

0.33 

Alternative  3 

1.4 

5 

0.56 
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Contributors  to  Capability:  COIL  beam 

Capability  assumptions 

-  Published  “achievable”  COIL  beam  power  of  3  MW 

Development  assumptions 

-  Normalized  TRL  indicates  initial  readiness-level 

•  Determines  probability  of  disruptions  during  development 

-  Published  TRL  level  of  4  for  a  power  of  3  MW 


COIL  beam 
Alternative 

Power 

[MW] 

TRL  level 

Initial  Readiness-Level 
[m°(i,r)\ 

Alternative  1 

3 

4 

0.44 

Alternative  2 

4 

3 

0.33 

Alternative  3 

5 

1 

0.11 
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System  Risk  and  Interdependencies 


•  Candidate  families  of  systems  can  have  different  combinations  of  system- 
risk  and  interdependency  strengths 
-  These  characteristics  have  different  impact  on  development  success 


Strongly  dependent  systems 


Independent  systems 

maximum  inherent  risk,  a. 


dependency  strength,  S(i,j) 


expected  implementation  time  [time-steps] 
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System  Risk  and  Interdependencies 


•  Candidate  families  of  systems  can  have  different 
combinations  of  system-risk  and  interdependency  strengths 

•  These  characteristics  have  different  impact  on  development  success 


Strongly  dependent  systems 


0.15 


0.05 


maximum  inherent  risk, 


sys 


sys 


sys 


0.6 


dependency  strength,  S(i,j) 


10  20  30  40 

Implementation  time-step 
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Capability  Modeling 


Assume  desired  ABL  capability  to 
“disable  threat  from  600  km  (slant 
range)” 

-  Categories  of  systems  create 
different  capability  levels 

Identify  functions  that  comprise 
capability 

Identify  systems  that  perform  each 
function 

First-order  quantification  of 
capability 

-  Aircraft  system  indirectly 
considered  (host  of  other 
systems) 


Defense  Industry  Daily,  2009 


Disable  missile 
from  600  km 


R 


Detect  & 

Engage 

Disable 

Track  Missile 

Missile 

Missile 

Detection  & 

COIL 

Aircraft 

AO 

Tracking  System 

Beam 

System 

D,  P,  X 


bn,  Sffi 
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